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(57) Abstract: A motion compensated video coding 
method which can be applied especially in transfer 
of video streams using low transmission bit rate is 
presented. In the motion compensated coding method, 
the motion of picture elements between a piece of 
reference video information and a piece of current 
video information is estimated and then modeled using 
certain basis function and coefficients. The coefficients 
are quantized, and the quantizer is selected according 
to a certain selection criterion, for example, based 
on a target image quality or on a target dansmission 
bit rate. Preferably the selection criterion is such 
that it automatically adjust the accuracy with which 
the motion of picture elements is represented to be 
related to the accuracy with which the prediction 
error information is represented. A decoding method, 
an encoder and a corresponding decoder are also 
described. 
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Method for encoding and decoding video information, a motion compensated 
video encoder and a corresponding decoder 



5 The present invention relates to video coding. In particular, it relates to compression 
of video information using motion compensated prediction. 

Background of the invention 

10 A video sequence consists of a large number video frames, which are formed of a 
large number of pixels each of which is represented by a set of digital bits. Because 
of the large number of pixels in a video frame and the large number of video frames 
even in a typical video sequence, the amount of data required to represent the video 
sequence quickly becomes large. For instance, a video frame may include an array 

15 of 640 by 480 pixels, each pixel having an RGB (red, green, blue) color 
representation of eight bits per color component, totaling 7,372,800 bits per frame. 
Video sequences comprise a sequence of still images, which are recorded/displayed 
at a rate of typically 15 - 30 frames per second. The amount of data needed to 
transmit information about each pixel in each frame separately would thus be 

20 enormous. 

Video coding tackles the problem of reducing the amount of information that needs 
to be transmitted in order to present the video sequence with an acceptable image 
quality. For example, in videotelephony the encoded video information is 
25 transmitted using conventional telephone networks, where transmission bit rates are 
typically multiples of 64 kilobits/s. In mobile videotelephony, where transmission 
takes place at lesast in part over a radio communications link, the available 
transmission bit rates can be as low as 20 kilobits/s. 

30 In typical video sequences the change of the content of successive frames is to a 
great extent the result of the motion in the scene. This motion may be due to camera 
motion or due to motion of the objects present in the scene. Therefore typical video 
sequences are characterized by significant temporal correlation, which is highest 
along the trajectory of the motion. Efficient compression of video sequences usually 

35 takes advantage of this property of video sequences. Motion compensated 
prediction is a widely recognized technique for compression of video. It utilizes the 
fact that in a typical video sequence, image intensity/chrominance value in a 
particular frame segment can be predicted using image intensity/chrominance values 
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of some other already coded and transmitted frame, given the motion trajectory 
between these two frames. Occasionally it is advisable to transmit a whole frame, to 
prevent the deterioration of image quality due to accumulation of errors and to 
provide additional functionalities, for example, random access to the video 
5 sequence). 

A schematic diagram of an example video coding system using motion compensated 
prediction is shown in Figures 1 and 2 of the accompanying drawings. Figure 1 
illustrates an encoder 10 employing motion compensation and Figure 2 illustrates a 
10 corresponding decoder 20. The operating principle of video coders using motion 
compensation is to minimize the prediction error frame E„(x,y), which is the 
difference between the current frame I n (x,y) being coded and a prediction frame 
p n ( x > y) • The prediction error frame is thus 

is E n (x,y)=i n (x,y)-?n(x>y)- (i) 

The prediction frame is built using pixel values of a reference frame i^(x,j>) , which 
is one of the previously coded and transmitted frames (for example, a frame 
preceding the current frame), and the motion of pixels between the current frame 

20 and the reference frame. The motion of the pixels may be presented as the values of 
horizontal and vertical displacements Ax(x 9 y) and Ay(x,y) of a pixel at location 
(x,y) in the current frame I n ( x >y)- The P 3 * 1 of numbers [Ax(x,y), Ay(*,J>)] is called 
the motion vector of this pixel. The motion vectors are typically represented using 
some known functions (called basis functions) and coefficients (this is discussed in 

25 more detail below), and an approximate motion vector field (Ax(x,j>), Ay(x,y)) can 
be constructed using the coefficients and the basis functions. 

The prediction frame is given by 

30 P n (x y y) = R n [x + Zx(x y y\y + Ay(x 9 y)\ (2) 

where the reference frame i^(x,.y) is available in the Frame Memory 17 of the 
encoder 10 and in the Frame memory 24 of the decoder 20 at a given instant An 
information stream 2 carrying information about the motion vectors is combined 
35 with information about the prediction error (1) in the multiplexer 16 and an 
information stream (3) containing typically at least those two types of infomation is 
sent to the decoder 20. 
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In the Prediction Error Coding block 14, the prediction error frame E n (x,y) is 
typically compressed by representing it as a finite series (transform) of some 2- 
dimensional functions. For example, a 2-dimensional Discrete Cosine Transform 
5 (DCT) can be used. The transform coefficients related to each function are 
quantized and entropy coded before they are transmitted to the decoder (information 
stream 1 in Figure 1). Because of the error introduced by quantization, this 
operation usually produces some degradation in the prediction error frame E n (x,y) . 

10 In the Frame Memory 24 of the decoder 20 there is a previously reconstructed 
reference frame B^ix.y). Using the decoded motion information (Ax(x,j/),Ay(x,j>)) 
and R n (x 9 y) it is possible to reconstruct the prediction frame P n (x 7 y) in the Motion 
Compensated Prediction block 21 of the decoder 20. The transmitted transform 
coefficients of the prediction error frame E n (x 9 y) are used in the Prediction Error 

15 Decoding block 22 to construct the decoded prediction error frame E n (x 9 y) . The 
pixels of the decoded current frame I„(x 9 y) are reconstructed by adding the 
prediction frame P n (x 9 y) and the decoded prediction error frame E n (x 7 y) 

I n (x 9 y) = P n (x 9 y) + E n (x 9 y) = R n [x + Ax(x,y),y + Ay(x 9 yj\+E n (x,y). (3) 

20 

This decoded current frame may be stored in the Frame Memory 24 as the next 
reference frame jR^, (x 9 y) . 

Let us next discuss in more detail the motion compensation and transmission of 
25 motion information. The construction of the prediction frame P n (x 9 y) in the Motion 
Compensated Prediction block 13 of the encoder 10 requires information about the 
motion in the current frame I n (x 9 y). Motion vectors [Ax(x 9 y) 9 Ay(x 9 y)] are calculated • 
in the Motion Field Estimation block 11 in the encoder 10. The set of motion 
vectors of all pixels of the current frame [Ax(),Ay()] is called the motion vector 
30 field. Due to the very large number of pixels in a frame it is not efficient to transmit 
a separate motion vector for each pixel to the decoder. Instead, in most video coding 
schemes the current frame is divided into larger image segments and information 
about the segments is transmitted to the decoder. 

35 The motion vector field is coded in the Motion Field Coding block 12 of the 
encoder 10. Motion Field Coding refers to representing the motion in a frame using 
some predetermined functions or, in other words, representing it with a model. 

»■ 
I 
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Almost all of the motion vector field models commonly used are additive motion 
models. Motion compensated video coding schemes may define the motion vectors 
of image segments by the following general formula: 

5 Ax(x, JO = 2 a ifi( x >y) (4) 

M-l 

4y(*>;>) = 5>,a(*,j0 (5) 

where coefficients a. and b t are called motion coefficients. They are transmitted to 
10 the decoder. Functions f t and g { are called motion field basis functions, and they 
are known both to the encoder and decoder. 

In order to minimize the amount of information needed in sending the motion 
coefficients to the decoder, coefficients can be predicted from the coefficients of the 

15 neighboring segments. When this kind of motion field prediction is used, the motion, 
field is expressed as a sum of a prediction motion field and refinement motion field. 
The prediction motion field uses the motion vectors associated with neighboring 
segments of the current frame. The prediction is performed using the same set of 
rules and possibly some auxiliary information in both encoder and decoder. The 

20 refinement motion field is coded, and the motion coefficients related to this 
refinement motion field are transmitted to the decoder. This approach typically 
results in savings in transmission bit rate. The dashed lines in Figure 1 illustrate 
some examples of the possible information some motion estimation and coding 
schemes may require in the Motion Field Estimation block 11 and in the Motion 

25 Field Coding block 12. 

Polynomial motion models are a widely used family of models. (See, for example 
H. Nguyen and E. Dubois, "Representation of motion information for image 
coding," in Proc. Picture Coding Symposium '90, Cambridge, Massachusetts, March 

30 26-18, 1990, pp. 841-845 and Centre de Mbrphologie Mathematique (CMM), 
"Segmentation algorithm by multicriteria region merging," Document SIM(95)19, 
COST 2 liter Project Meeting, May 1995). The values of motion vectors are 
described by functions which are linear combinations of two dimensional 
polynomial functions. The translational motion model is the simplest model and 

35 requires only two coefficients to describe the motion vectors of each segment The 
values of motion vectors are given by the formulae: 
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Ax(x,y) = a 0 
Ay(x,j/) = 5 0 



(6) 



This model is widely used in various international standards (ISO MPEG-1, MPEG^ 
5 2, MPEG-4, ITU-T Recommendations H.261 and H.263) to describe motion of 
16x16 and 8x8 pixel blocks. Systems utilizing a translational motion model 
typically perform motion estimation at full pixel resolution or some integer fraction 
of full pixel resolution, for example with an accuracy of 1/2 or 1/3 pixel resolution. 

10 Two other widely used models are the affine motion model given by the equation: 
Ay(x,y) = b 0 +b l x+b 2 y 

and the quadratic motion model given by the equation: 
Ax(x 9 y) = a 0 + a l x + a 2 y+a z xy+a 4 x 2 +a 5 y 2 

Ay(x 9 y) = * 0 + b i x + T^y + h*y + K* 2 + b sy 2 



The affine motion model presents a very convenient trade-off between the number 
of motion coefficients and prediction performance. It is capable of representing 

20 some of the common real-life motion types such as translation, rotation, zoom and 
shear with only a few coefficients. The quadratic motion model provides good 
prediction performance, but it is less popular in coding than the affine model, since 
it uses more motion coefficients, while the prediction performance is not 
substantially better. Furthermore, it is computationally more costly to estimate the 

25 quadratic motion than to estimate the affine motion. 

When the motion field is estimated using higher order motion models (such as 
presented, for example, in equations 7 and 8), the motion field estimation results in 
a motion field represented by real numbers. In this case the motion coefficients need 
30 to be quantized to a discrete accuracy before they are transmitted to the decoder. 

The Motion Field Estimation block 11 calculates motion vectors [Ax(x 9 y) 9 Ay(x 9 y)]i 
of the pixels of a given segment which minimize some measure of prediction 
error in the segment. In the simplest case the motion field estimation uses the 
35 current frame I n (x 9 y) and the reference &amcR n (x 9 y) as input values. Typically the 
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Motion Field Estimation block outputs the motion field [Ax(x,y),Ay(x,y)] to the 
Motion Field Coding block 12. The Motion Field Coding block makes the final 
decisions on what kind of motion vector field is transmitted to the decoder and how 
the motion vector field is coded. It can modify the motion model and motion 
5 coefficients in order to minimize the amount of information needed to describe a 
satisfactory motion vector field. 

The image quality of transmitted video frames depends on the accuracy with which 
the prediction frame can be constructed, in other words on the accuracy of the 

10 transmitted motion information, and on the accuracy with which the prediction error 
information is transmitted. Here the term accuracy refers not only to the ability of 
the emotion field model to represent the motion within the frame but also to the 
numrical precision with which the motion information and the prediciton error 
information is represented. Motion information transmitted with hich accuracy may 

15 be canceled out in the decoding phase due to low accuracy of the precidiction error 
frame, or vice versa. 

Current video coding systems employ various motion estimation and coding 
techniques, as discussed above. The accuracy of the motion information and the 
20 transmission bit rate needed to transmit the motion information are typically 
dictated by the choice of the motion estimation and coding technique, and a chosen 
technique is usually applied to a whole video sequence. Generally, as the accuracy 
of the transmitted motion information increases, the amount of transmitted 
information increases. 

25 ; 
In general, better image quality requires larger amounts of transmitted information. 
Typically, if the available transmission bit rate is limited, this limitation dictates the 
best possible image quality of transmitted video frames. It is also possible to aim for 
a certain target image quality, and the transmission bit rate then depends on the 

30 target image quality. In current video coding and decoding systems, the trade-offs 
between the required transmission bit rate and image quality are mainly made by 
adjusting the accuracy of the information presenting the prediction error frame. This 
accuracy may change, for example, from frame to frame, or even between different 
segments of a frame. 

35 

The problem in changing the accuracy of the transmitted prediction error frame is 
that it may cause unexpected degradation of the overall performance of the video 
encoding, for example, when conforming to a new available transmission bit rate. In 
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other words, the image quality achieved is not as good as that expected considering 
the transmission bit rate. The image quality may deteriorate drastically, when a 
lower transmission bit rate is available, or the image quality may not be enhanced 
even though a higher transmission bit rate is used. 

5 

SUMMARY OF THE INVENTION 

The object of the invention is to provide a flexible and versatile motion 
compensated method for encoding/decoding video information. A further object of 
10 the invention is to provide a method that ensures good transmitted video quality for 
various transmission bit rates. A further object is that the method may employ 
various motion estimation and coding techniques. 

These and other objects of the invention are achieved by selecting the quantization 
15 accuracy of the motion coefficients so that the accuracy of the transmitted motion 
information is compatible with the accuracy of the prediction error information. ; 

A method according to the invention is a method for encoding video information, 
comprising the following steps of: 
20 - estimating the motion of picture elements between a piece of reference video 
information and a piece of current video information, 

- modeling the motion of picture elements using a certain set of basis functions and 
certain motion coefficients, 

- defining a certain set of quantizers, 

25 - selecting, based on a certain predetermined selection criterion, a motion 
coefficient quantizer from the set of quantizers, and 

- quantizing the motion coefficients using the selected motion coefficient quantizer. 

In a method according to the invention, the motion of picture elements between a 
30 certain piece of reference video information and a piece of current vided 
information is estimated The resulting motion vector field is represented using 
certain basis functions and motion coefficients. The basis functions are known both 
to the encoder and decoder, so the transmission of said coefficients enables the 
decoder to determine an estimate for the motion of picture elements. Typically the 
35 coefficient values are real numbers, and therefore quantization is needed in order to 
present the coefficients to a certain discrete accuracy using a certain number of bits. 
The coefficients are quantized before transmission to the decoder, or before using 
them in constructing a prediction frame. 
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In a method according to the invention, a set of quantizers is provided. Here the 
term quantizer refers to a function mapping real numbers to certain reconstruction 
values. For each reconstruction value there is a quantization interval determining the 
5 range of real numbers which are mapped/quantized to said reconstruction value. The 
size of the quantization intervals can be, for example, the same for each 
reconstruction value (uniform quantizer) or the size of the quantization interval can 
be different for each reconstruction value (non-uniform quantizer). The quantization 
interval determines the accuracy with which the coefficients are represented. The 
10 quantizers in the set may all be similar so that the reconstruction values and the 
quantization intervals are scaled from quantizer to quantizer using a certain 
parameter. It is also possible that the set of quantizers comprises different types of 
quantizer, both uniform and non-uniform, for example. Quantizers are further 
.discussed in the detailed description of the invention. 

15 

The selection criterion for the quantizer in a method according to the invention can 
be, for example, the target image quality or the target transmission bit rate. The 
selection of a quantizer may also be bound to some other variable, which depends 
on the target image quality or the transmission bit rate. A new quantizer can be 
20 selected, for example, each time the target transmission bit rate changes. For various 
parts of a video frame, for example, it is possible the use different quantizers. \ 

) - 

According to the invention, when the target image quality changes, it is possible to 
adjust the accuracy with which both the prediction error information and the motion 

25 information is encoded and transmitted. Therefore, for each image quality or 
transmission bit rate, it is possible to obtain good overall coding performance. It is 
possible, for example, to adjust both the accuracy of the prediction error 
information and that of the motion information to be able to transmit the encoded 
video stream using a certain bit rate. It is also possible, for example, that the 

30 accuracy of the prediction error information is dictated by the target image quality, 
and in the method according to the invention the accuracy of the motion information 
is adjusted to be compatible with the prediction error accuracy: quantization should 
not be too fine, because the motion information cannot enhance image quality 
beyond the accuracy provided by the prediction error information, but not too 

35 coarse either, because coarsely quantized motion information may deteriorate the 
image quality provided by the prediction error information. • 
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A method according to the invention does not pose any restrictions on the motion 
field estimation or motion field coding techniques used to obtain the motion 
coefficients. It is therefore applicable with any such techniques. For example, 
motion model adaptation may be used to avoid the cost of using an over accurate 
5 motion field estimation and coding technique by providing a selection of motion 
field estimation and/or coding techniques providing various accuracies. An 
appropriate motion field estimation and/or coding technique can then be selected 
based on the target image quality or target bit rate, and fine-tuning between the 
prediction error information accuracy and the motion information accuracy can be 
10 performed by selecting a proper quantizer. 

The invention can be straightforwardly applied to existing motion compensated 
video coding methods and systems. In such prior-art systems, the quantization of. 
motion coefficients is typically done to a certain, predetermined accuracy, which 
15 works well for a certain target image quality. When the target image quality or the 
available transmission bit rate differs remarkably from the designed image quality, 
the video encoder produces worse image quality than that which could be achieved 
for a given transmission bit rate. This effect can be eliminated by selecting a more 
appropriate quantizer for the motion coefficients according to the invention. 

20 

The invention also relates to a method for decoding encoded video information, 
comprising the following steps of: 

- receiving quantized motion coefficients describing motion of picture elements, 

- defining a set of inverse quantizers, 

25 - determining a selected motion coefficient quantizer using which the motion 
coefficients are quantized, 

- performing inverse quantization of the quantized motion coefficients using an 
inverse quantizer corresponding to the selected motion coefficient quantizer, 

- determining the motion of the picture elements using the inverse quantized motion 
30 coefficients and certain basis functions, and 

- determining a piece of prediction video information using a piece of reference 
video information and the determined motion of the picture elements. 

An encoder according to the invention is an encoder for performing motion 
35 compensated encoding of video information, comprising: 

- means for receiving a piece of current video information, 

- memory means for storing a piece of reference video information, 
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- motion field estimation means for estimating a motion field of picture elements in 
the piece of current video information using at least the piece of reference videq 
information, 

- motion field coding means, which comprise 

5 - means for producing motion coefficients describing the estimated motion 

field, 

- first selection means for selecting a quantizer from a set of quantizers, said 
first selection means having an input to receive information indicating a 
selection criterion and an output to send information indicating the selected 

10 quantizer, and 

- quantization means for quantizing motion coefficients using the selected 
quantizer, said quantization means having an input to receive information 
indicating the selected quantizer, a second input to receive the motion 
coefficients, and an output to send the quantized motion coefficients, and 

15 - motion compensated prediction means, which means comprise 

- second selection means for selecting an inverse quantizer from a set of 
inverse quantizers, said second selection means having an input to receive 
information indicating a selection criterion and an output to send information 
indicating the selected inverse quantizer, 

20 - inverse quantization means for inversely quantizing the quantized motion 

coefficients using the selected inverse quantizer, said quantization means 
having an input to receive the quantized motion coefficients, a second input to 
receive information indicating the selected inverse quantizer and an output to 
send the inverse quantized motion coefficient, and 
25 - means for determining a a piece of prediction video information using at least 
the piece of reference video information and the inverse quantized motion 
coefficients. 



The invention relates further to a decoder for performing the decoding of encoded 
30 video information, comprising: 

- memory means for storing a piece of reference video information, 

- input means for receiving quantized motion coefficients, and 

- motion compensated prediction means, which comprise 

- selection means for selecting an inverse quantizer from a set of inverse 
35 quantizers, said second selection means having an input to receive information 

indicating a selection criterion and an output to send information indicating the 
selected inverse quantizer, 
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- inverse quantization means for inversely quantizing the quantized motion 
coefficients using the selected inverse quantizer, said quantization means 
having an input to receive the quantized motion coefficients, a second input to 
receive information indicating the selected inverse quantizer and an output to 

5 send the inverse quantized motion coefficient, and 

- prediction means for determining a piece of prediction video information 
using at least the piece of reference video information and the inverse 
quantized motion coefficients. 

10 In one advantageous embodiment of the invention, the encoder and decoder are 
combined to form a codec. The motion compensated prediction parts of the encoder 
and decoder are similar, and they may be provided by a common part, which is 
arranged to operate as a part of the encoder and a part of the decoder, for example, 
alternatingly. 

15 

The invention relates further to a computer program element for performing motion 
compensated encoding of video information, comprising: 

- means for receiving a piece of current video information, 

- memory means for storing a piece of reference video information, 

20 - motion field estimation means for estimating a motion field of picture elements in 
the piece of current video information using at least the piece of reference video 
information, • 

- motion field coding means, which comprise 

- means for producing motion coefficients describing the estimated motion 
25 field, 

- first selection means for selecting a quantizer from a set of quantizers, said 
first selection means having an input to receive information indicating a 
selection criterion and an output to send information indicating the selected 
quantizer, and 

30 - quantization means for quantizing motion coefficients using the selected 

quantizer, said quantization means having an input to receive information 
indicating the selected quantizer, a second input to receive the motion 
coefficients, and an output to send the quantized motion coefficients, and 

- motion compensated prediction means, which means comprise 

35 - second selection means for selecting an inverse quantizer from a set of 

inverse quantizers, said second selection means having an input to receive 
information indicating a selection criterion and an output to send information 
indicating the selected inverse quantizer, 
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- inverse quantization means for inversely quantizing the quantized motion 
coefficients using the selected inverse quantizer, said quantization means 
having an input to receive the quantized motion coefficients, a second input to 
receive information indicating the selected inverse quantizer and an output to 

5 send the inverse quantized motion coefficient, and 

- means for determining a piece of prediction video information using at least 
the piece of reference video information and the inverse quantized motion 
coefficients. 

10 A second computer program element according to the invention is a computer 
program element for performing the decoding of encoded video information, 
comprising: : 

- memory means for storing a piece of reference video information, 

- input means for receiving quantized motion coefficients, and 
15 - motion compensated prediction means, which comprise 

- selection means for selecting an inverse quantizer from a set of inverse 
quantizers, said second selection means having an input to receive information 
indicating a selection criterion and an output to send information indicating the 
selected inverse quantizer, 

20 - inverse quantization means for inversely quantizing the quantized motion 

coefficients using the selected inverse quantizer, said quantization means 
having an input to receive the quantized motion coefficients, a second input to 
receive information indicating the selected inverse quantizer and an output to 
send the inverse quantized motion coefficient, and 

25 - prediction means for determining a piece of prediction video information 

using at least the piece of reference video information and the inverse 
quantized motion coefficients. 

According to one advantageous embodiment of the invention, a computer program 
30 element as specified above is embodied on a computer readable medium. 

The novel features which are considered as characteristic of the invention are set 
forth in particular in the appended Claims. The invention itself, however, both as to 
its construction and its method of operation, together with additional objects and 
35 advantages thereof^ will be best understood from the following description of 
specific embodiments when read in connection with the accompanying drawings. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 illustrates an encoder for motion compensated coding of video according 
to prior art, 

5 

Figure 2 illustrates a decoder for motion compensated coding of video according 
to prior art, 

Figure 3 illustrates a set of uniform quantizers, 

10 

Figure 4 illustrates a modified uniform quantizer, 

Figure 5 illustrates a flowchart of a motion compensated video encoding method 
according to the invention, 

15 

Figure 6 illustrates a flowchart of a motion compensated video decoding method 
according to the invention, 

Figure 7 illustrates the motion field encoding and motion compensated prediction 
20 blocks of an encoder according to a first preferred embodiment of the 

invention, 



25 



Figure 8 illustrates an encoder for motion compensated encoding of video 
according to a second preferred embodiment of the invention, 

Figure 9 illustrates a decoder for motion compensated decoding of video 
according to the second preferred embodiment of the invention, and 



Figure 10 illustrates a motion field encoding block of an encoder according to a 
3 0 third preferred embodiment of the invention. 

DETAHXED DESCRIPTION 

Figures 1 and 2 are discussed in detail in the description of motion compensated 
35 video encoding and decoding according to prior art. 



In encoding and decoding methods according to the invention, a set of quantizers is 
defined. Different quantizers may differ in the way reconstruction values and 
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intervals are selected. Quantization of a motion coefficient a t can be performed e.g. 
by utilizing following equation: 



(9) 



10 



where q is a quantization parameter defining the interval between reconstruction 
points and [x J is the 'floor 1 operation (largest integer not larger than x). In this case, 
the quantization output a i is an integer index that can be entropy coded and 
transmitted to a decoder. In this example, the corresponding inverse quantization 
operation can be defined by 



(10) 



and the inverse quantized motion coefficients a i are used to approximate the 
15 original coefficients a i . Equations 9 and 10 define a uniform quantizer with evenly 
distributed reconstruction values and reconstruction intervals of equal size. Figure 3 
illustrates three uniform quantizers having different values of q: q u qi and q$. 

Figure 4 illustrates a modified uniform quantizer. In this quantizer, parameter q 
20 defines the reconstruction points. The quantization interval, in which the coefficient 
at values are mapped to <S, = 0, is larger than the other quantization intervals, which 
are of constant size. This quantizer can be defined, for example, using an additional 
parameter s 3 which is half of the increase in the quantization interval for d§ = 0. 
Quantization according to the modified uniform quantizer can* be defined, for 
25 example, by 



aj-sgnta)*- 



30 



where sgn(a/) gives the sign of a t . The corresponding inverse quantization operation 
can be defined, for example, using equation 10. 
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When the modified uniform quantizer is used, the transmission of quantized 
coefficients d% is likely to require fewer bits. This is because the quantization 
interval corresponding to values near zero is larger than in the uniform quantizer, 
and typically a zero-valued quantized coefficient can be transmitted using a smaller 
5 number of bits than quantized coefficients having non-zero values. 

Above, two different quantizers are described as examples of quantizers that can be 
used in a method according to the invention. In quantization, the size of the 
quantization interval is inversely proportional to the number of bits needed to 

10 represent the quantized coefficients d f . The quantization interval also determines the 
accuracy according to which the inverse quantized coefficients a/ correspond to the 
original coefficients a*. The quantization interval thus represents the trade-off 
between quantization accuracy and the amount of information needed in 
transmitting motion coefficients to the decoder. In a method according to the 

15 invention, a set of quantizers is defined for quantizing the motion coefficients. The 
set may comprise, for example, various uniform quantizers having different values 
of q. 

Figure 5 presents a flowchart of a motion compensated video encoding method 
20 according to the invention. In step 501, a set of quantizers for quantizing the motion 
coefficients is defined. The quantizers may be, for example, uniform quantizers 
having various quantization intervals, that is different values of parameter q. When 
uniform quantizers are used, it is enough to define the allowed quantization 
intervals in step 501. In step 502 an appropriate criterion for selecting the quantizer 
25 is defined. This criterion can be, for example, the target image quality or the target 
bit rate. It is also possible to select the motion information quantizer based on a 
previously selected prediction error quantizer. In step 503 the current frame iw(xj>) 
is received. In step 504 motion in the current frame is estimated and modeled, fps 
example, segment by segment using certain basis functions fi(xj>) and gfcd?) 9 and 
30 coefficients a t and b t . The motion vector field is thus represented as 

1=0 
M-l 

f=0 

35 Selecting the quantizer according to the invention does not pose any restrictions on 
the motion field estimation and/or coding methods that can be used. The estimation 
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and coding of the motion vector field can be made, for example, using an^ 
techniques used in prior art motion compensated video coding schemes. The basis 
functions used in coding the motion field vector are most preferably orthogonal, 
because orthogonal functions are less sensitive to inaccuracies in the decoded 
5 motion coefficients caused by quantization. Any basis function, however, may be 
used. Furthermore, the motion vector field that is coded can be, for example, a 
refinement motion vector field which is used to correct a prediction motion vector 
field constructed, for example, by using the motion of neighboring segments within 
the frame. 

10 

When a translational model is used to represent the motion vector field, the 
estimation of the motion vector field is typically carried out using a certain 
accuracy. This accuracy is the upper limit (that is, the best possible accuracy) with 
which the motion coefficients do and b 0 can be determined. Using quantization 
15 according to the invention, it is possible to lower the accuracy with which the 
motion coefficients are transmitted to a receiver. 

In step 505 the quantizer to be used in die quantization of the motion coefficients a t 
and bi is selected and in step 506 the motion coefficient are quantized, resulting in 

20 the quantized coefficients a. and S t . They are transmitted to a receiver in step 511. 
Advantageously the receiver determines the motion information quantizer with 
reference to, for example, the quantizer (QP) used to encode the prediction error 
information or the target bit rate of the encoded video stream. If the motion 
coefficient quantizer is chosen with reference to the quantizer QP used to encode 

25 the prediction error information, the receiver is able to determine the motion 
coefficient quantizer implicitly. In this case an explicit indication of the selected 
value of q 9 for example, is not required. It is, however, an alternative to transmit 
information relateing to q in step 512. It can be transmitted as part of the encoded 
video frame or it can be signaled independently to the receiver. Furthermore, the 

30 receiver may determine the selected quantizer, for example, from the format ir£ 
which the quantized coefficients are transmitted. ■ 

In step 507 a prediction frame P w (Xy) can be determined using a reference frame 
Rn(x,y) 

35 

P n (*,y) = K (* + M*>y)>y + Ay(*,.y)) . 



WO 01/54417 PCT/FI01/00044 

17 

Inverse quantized coefficients a i and are typically used in constructing 
Ax(^3 / ) = 2 ff i/i( 3C »J ; ) an d &y(x i y) = '£ t b i g i (x 9 y). Thereafter the prediction error 
frame E n (x 3 y) = I n (x,yyP n (xji) can be determined (step 508), and it can be encoded 
(step 509). After encoding, the prediction error frame can be represented using 
5 prediction error coefficients c h In step 510 the prediction error coefficients are 
quantized using a second quantizer. The resulting quantized prediction error 
coefficients c. are transmitted to the receiver in step 511. The selection of the 
second quantizer is out of the scope of the present invention. Typically its selection 
is dictated by the target image quality. It may alternatively be defined according to 
10 the requirements of a video coding standard, for example ITU-T Recommendation 
H.261orH263. 

Above, frames as used as an example of a piece of video information. In encoding 
and decoding methods and apparatus according to the invention, it is also possible 
15 to transmit and process video information, for example, block-by-block or segment- 
by-segment in addition to the frame-by-frame approach. 

Figure 6 illustrates a flowchart of a decoding method for motion compensated coded 
video frames according to the invention. In step 601 a set of inverse quantizers is 

20 defined, and in step 602 the quantized coefficients a i and b t are received. In step 
603 the motion information quantizer is determined. It can be determined, for 
example, by deteimining from the received prediction error coding data tile., 
prediction error quantizer, and selecting an inverse quantizer for motion coefficients 
according to a predefined set of rules. Alternatively, the inverse quantizer may be 

25 selected according to an explicit indication in the received bit stream. Thereafter 
inverse quantization of the motion coefficients is carried but (step 604), resulting in 
the inverse quantized motion coefficients a t and b { . In step 605, the basis function 
fi(xy) and &(*,)>) are determined. The basis function may be determined a priori if 
the motion estimation and coding technique uses only certain basis functions. It is 

30 also possible that the motion estimation and coding technique uses motion model 
adaptation, in which case information about the selected basis functions is 
transmitted to the receiver.Information about these can be, for example, transmitted 
within the encoded frame, or the basis functions may be otherwise known to both 
the encoding and decoding method. 

35 

In step 606 the motion of picture elements is determined using at least the inverse" 
quantized motion coefficients and the basis functions. Thereafter the predictiorf 
frame P n (xy) can be determined in step 607. In step 608 the quantized prediction 
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error coefficients c i are dequantized using an inverse quantizer related to the 
prediction error. In step 609 a decoded prediction error frame E n (x,y) is 
constructed using the inverse quantized prediction error coefficients c t . In step 610 
the decoded current frame is determined, for example, by 
5 I n (x 9 y) = P n (x,y) + E n (x,y). 

Figure 7 presents a schematic drawing of a motion field coding block and a motion 
compensated prediction block of a motion compensated video encoder according to 
a first preferred embodiment of the invention. In the motion field coding block 12 f 

10 motion field coding is carried out using the motion vector field coding block 51. 
The output of the motion vector field coding block comprises the motion 
coefficients a t and b t . The quantization block 30 is responsible for the quantization 
of the motion coefficients. It performs the quantization using a selected quantizer. _ 
31. The quantizer selection block 32 comprises a set of quantizers 33, from which 

15 the currently used quantizer is selected. The quantizer selection criterion block 34 
outputs the current criterion for the selection. It may, for example, indicate the 
current target image quality or the target bit rate, and the quantizer selection block 
32 selects an appropriate quantizer based on the criterion. 

20 In the motion compensated prediction block 13 f according to the invention, there is 
an Inverse Quantization Selection block 42, comprising a set of inverse quantizers 
43. Inverse quantization is carried out in the Inverse Quantization block 40, using 
the selected inverse quantizer 41. The inverse quantized motion coefficients a t and 
ijare typically further used in the Motion Vector Field Decoding block 52 for 

25 determining a prediction frame 

Information about the quantized motion coefficients is transmitted from the 
Quantization block 30 to the Inverse Quantization block 40 in the Motioii 
Compensated Prediction block 13'. In Figure 7, information about the selected 
30 quantizer is transmitted from the Quantizer Selection block 32 to the Inverse 
Quantizer Selection block 42 in the Motion Compensated Prediction block 13' in 
the encoder. 

A decoder according to the invention comprises a Motion Compensated Prediction 
35 block 21' which is similar to the Motion Compensated Prediction block 13' in an 
encoder according to the invention. The Inverse Quantization Selection block 42 in 
the decoder may receive separate information from which it can determine the 
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correct inverse quantizer from the encoder or, alternatively, it may receive the 
information from some other part of the decoder. 

Figure 8 presents a schematic diagram of a motion compensated video encoder 
5 according to a second preferred embodiment of the invention. In the encoder 10' 
according to the invention there is Motion Field Coding block 12', which comprises 
a Quantization block 30 and a Quantizer Selection block 32. The Motion 
Compensated Prediction block 13' of the encoder 10 f comprises an Inverse 
Quantization block 40 and an Inverse Quantizer Selection block 42. The quantized 
10 coefficients (arrow 2) and information about the prediction error (arrow 1) are 
transmitted to a decoder. 

In an encoder according to the second preferred embodiment of the invention, the 
selection of the motion coefficient quantizer is dictated by the quantizer QP used in 

15 the Prediction Error Coding block 14 of the encoder. The prediction error quantizer 
QP can be selected, for example, based on the target image quality. By binding the 
selection of the quantizer used in quantizing the motion coefficients to the quantizer 
used in the coding of the prediction error, the target image quality of the encoded, 
video frames can be reliably attained. The Prediction error Coding block 14 may 

20 indicate the current prediction error quantizer (QP) to the Quantizer Selection block 
32 and also to the Inverse Quantizer selection block 42 in the encoder 10'. The 
Quantizer Selection block 32 may alternatively inform the Inverse Quantization 
Selection block 42 in the encoder 10' about the prediction error quantizer. 

25 The Quantizer Selection block 32 and the Inverse Quantizer Selection block 42 may 
alternatively reside, for example, in the Prediction Error Coding block 14 of the 
encoder. 

In a decoder accoding to the second preferred emdobiment of the invention, the 
30 decoder receives information about the quantizer used in coding the prediction error 
frame and thus there usually is no need to transmit explicit information concerning 
the quantizer of the motion coefficients. The decoder may determine the motioif 
coefficient quantizer using the information transmitted from the Prediction Erxo? 
Coding block 14 (arrow 1) or from corresponding signaling information. The 
35 information streams 1 and 2 are typically multiplexed to an information stream 3 
representing the encoded video frame. This information stream is transmitted to a 
decoder according to the invention. The information about the prediction error 
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quantizer may alternatively reside in the decoder and require no communication 
between the encoder and decoder. 

The dashed arrows pointing to the Motion Field Coding blockl2' in Figure 8 
5 present some possible examples of information streams within the encoder. The 
details of the structure of the encoder depend, on the particular motion field 
estimation and coding techniques employed. 

Figure 9 presents a schematic diagram of a decoder 20 f for decoding motion 
10 compensation coded video frames according to the second preferred embodiment of 
the invention. It comprises a Motion Compensated Prediction block 21', where there 
is an Inverse quantization block 40 according to the invention and an Inverse 
Quantizer Selection block 42 according to the invention. Typically the Prediction 
Error Decoding block 22' informs the Inverse Quantizer Selection block 42 about 
15 the selected quantizer QP used to quantize the prediction error information. This 
Inverse Quantization Selection block 42 then selects an appropriate inverse 
quantizer for the motion coefficient information on the basis of the QP value it 
receives from the Prediction Error Decoding block. The Inverse Quantizer Selection 
block 42 may alternatively reside, for example, within the Prediction Error 
20 Decoding block 22 of the decoder. 

i 

Figure 10 presents a more detailed schematic diagram of a Motion Field Coding 
block 12* according to a third preferred embodiment of the invention. The main 
idea in this motion field coding block is to present the motion vector field as a sum 

25 of two vector fields: a vector field that can be predicted from other segments in the 
frame Ax^ (x,y) and Ay^ (x,y) and a refinement motion vector field Ax nEmmt (x 9 y) 
AVxcfiDcmcatC^^) • We refer to the patent application US 09/371641 concerning 
details of the motion vector field estimation and coding in a Motion Field Coding 
block 12* according to the third preferred embodiment of the invention. Here we 

30 only briefly describe the coding of the motion vector field in block 12* at a general 
level. 

In block 12* there is a Motion Analyser block 61 and a block 62, which is 
responsible for the selection of a neighboring segment and for building the 
35 Prediction Motion Field. Typically, when predicting the motion of a certain 
segment, the motion of at least one of its neighboring segments is used. This 
prediction is then refined by determining a refinement motion vector field, which is 
coded, for example, using an affine model employing orthogonal basis functions. 
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The Motion Coefficient Removal block 63 basically determines, which refinement 
motion coefficients can be set to zero without causing the accuracy of the motion 
vector field to deteriorate to an unacceptable level. The refinement motion 
coefficients, remaining after the removal procedure, are quantized in a Quantization 
block 30 according to the invention. 



10 



The selection of the quantizer used to quantize motion coefficients can be dictated 
by the choice of the quantizer used in the Prediction Error Coding block of the 
encoder. The Prediction Error Coding can be performed according to the ITU-T 
Recommendation H.263 using 2-dimensional Discrete Cosine Transform and 
defining two uniform quantizer QP for the frame: QP P stays constant over the whole 
frame and QP S may vary for each 16 x 16 pixel segment in the frame. 



15 



20 



In the third preferred embodiment of the in invention, the set of quantizers consists 
of uniform quantizers having different quantization intervals. Therefore, the selected 
motion coefficient quantizer can be specified by its quantization interval. Thie 
quantization interval, in turn, is defined by parameter q. This parameter can be 
dependent, for example, on the quantization parameter QP P . In Table 1, q denotes 
the quantization interval in the motion coefficient quantization and QP P denotes half 
of the quantization interval of the prediction error coefficients. Motion coefficients 
in block 12* can be quantized according to Table 1, in other words, by selecting the 
quantizer in block 30 based on the value of QP P . Thereafter the quantized motion 
coefficients can be entropy coded and transmitted to a decoder. 



25 Table 1. Relationship between the quantization interval q related to motion 
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As can be seen from Table 1, the size of the quantization interval used to quantize 
the motion coefficients is related to the quantization interval used in the 
quantization of the prediction error coefficients. This means that when the 
prediction error is transmitted using a higher accuracy, the motion information is 
5 automatically transmitted using a higher accuracy and vice versa. The value of QPp 
can be selected, for example, based on the target image quality. Once the value of 
QPp is known, it is possible straightforwardly to determine a quantizer and an 
inverse quantizer corresponding to said QP P value. 

10 It is also possible to have a set of quantizers which comprises uniform and non- 
uniform quantizers. In this case, it is also possible to select a quantizer depending 
on the value of QP P . 

An encoder or a decoder according to the invention can be realised using hardware 
15 or software, or using a suitable combination of both. An encoder or decoder 

implemented in software may be, for example, a separate program or a software. 

building block that can be used by various programs. The Quantization block, 

Quantizer Selection block and Quantizer Selection Criterion block are in the above 

description and in the drawings represented as separate units, but the functionality 
20 of these blocks can be implemented, for example, in one software program unit 

responsible for the selection of a quantizer and quantization of the motion 

coefficients. 

It is also possible to implement an encoder according to the invention and a decoder 
25 according to the invention in one functional unit. Such a unit is called a codec. A 
codec according to the invention may be a computer program or a computer 
program element, or it may impletemented at least partly using hardware. 

In view of the foregoing description it will be evident to a person skilled in the art 
30 that various modifications may be made within the scope of the invention. While £ 
number of preferred embodiments of the invention have been described in detail, it 
should be apparent that many modifications and variations thereto are possible, all 
of which fall within the true spirit and scope of the invention. 
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We claim: 

1. A method for encoding video information, comprising the following steps of: 

- estimating the motion of picture elements between a piece of reference video 
5 information and a piece of current video information, 

- modeling the motion of picture elements using a certain set of basis functions and 
certain motion coefficients, 

- defining a certain set of quantizers, 

- selecting, based on a certain predetermined selection criterion, a motion 
10 coefficient quantizer from the set of quantizers, and 

- quantizing the motion coefficients using the selected motion coefficient quantizer. 

2. A method for encoding video information according to claim 1, wherein the 
selection criterion is the value of a certain parameter used in the encoding. 

15 

3. A method for encoding video information according to claim 2, further 
comprising the following steps of: 

- defining a set of inverse quantizers, 

- determining a selected motion coefficient quantizer using which the motion 
20 coefficients are quantized, 

- performing inverse quantization of the quantized motion coefficients using an 
inverse quantizer corresponding to the selected motion coefficient quantizer, 

- determining the motion of the picture elements using the inverse quantized motion 
coefficients and the basis functions, 

25 - determining a piece of prediction video information using the piece of reference 
video information and the determined motion of the picture elements, 

- determining a piece of prediction error video information based on the difference 
of the piece of prediction video information and the piece of current video 
information, 

30 - coding the piece of prediction error video information and representing it with 
certain prediction error coefficients, 

- quantising the prediction error coefficients using a prediction error quantizer, and 

- selecting the motion coefficient quantizer based on the prediction error quantizer. 

35 4. A method for encoding video information according to claim 3, wherein the 
quantization interval of the motion coefficient quantizer is related to the 
quantization interval of the prediction error quantizer. 
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5. A method for encoding video information according to claim 1, wherein the 
predetermined selection criterion is the target image quality. 

6. A method for encoding video information according to claim 1, wherein the 
5 predetermined selection criterion is the amount of information needed to represent 

the quantized coefficients. 

7. A method for encoding video information according to claim 1, wherein the 
motion of picture elements is modeled using a set of orthogonal basis functions. 

10 

8. A method for encoding video information according to claim 7, wherein the 
motion of picture elements is modeled using a set of affine orthogonal basis 
functions. 

15 9. A method for encoding video information according to claim 7, wherein the 
motion of a picture element is represented by predicting the motion of the picture 
element based on the motion of certain neighboring picture elements and by 
detennining a refinement motion for the picture element 

20 10. A method for encoding video information according to claim 9, wherein the 
refinement motion is modeled using a set of affine orthogonal basis functions. 

11. A method for encoding video information according to claim 1, further 
comprising a step of transmitting the quantized motion coefficients to a receiver. 

25 

12. A method for encoding video information according to claim 11, furthef 
comprising a step of transmitting information specifying the selected motion 
coefficient quantizer to the receiver. 

30 13. A method for encoding video information according to claim 1, wherein the 
set of quantizers comprises a number of uniform quantizers each having a different 
quantization interval. 

14. A method for encoding video information according to claim 1, wherein the 
35 set of quantizers comprises a number of modified uniform quantizers, each having a 
different quantization interval. 



WO 01/54417 



PCT/FI01/00044 



25 

15. A method for decoding encoded video information, comprising the following 
steps of: 

- receiving quantized motion coefficients describing motion of picture elements, 

- defining a set of inverse quantizers, 

5 - determining a selected motion coefficient quantizer using which the motion 
coefficients are quantized, 

- performing inverse quantization of the quantized motion coefficients using an 
inverse quantizer corresponding to the selected motion coefficient quantizer, 

- determining the motion of the picture elements using the inverse quantized motion 
10 coefficients and certain basis functions, and 

- determining a piece of prediction video information using a piece of reference 
video information and the determined motion of the picture elements. 

16. A method for decoding encoded video information according to claim 15, 
15 further comprising a step of determining the basis functions using which the motion 

of the picture elements is modeled. ; 

17. A method for decoding encoded video information according to claim 15, 
wherein the selected motion coefficient (quantizer is determined from transmitted 

20 information relating to a certain parameter used in the encoding. 

18. A method for decoding encoded video information according to claim 16, 
wherein the received encoded video information comprises quantized prediction 
error coefficients describing a piece of prediction error video information, further 

25 comprising the following steps of: 

- determining a prediction error quantizer using which the prediction error 
coefficients are quantized, 

- performing inverse quantization of the quantized prediction error coefficients 
using an inverse quantizer corresponding to said prediction error quantizer, 

30 - determining a decoded piece of prediction error video information using the 
inverse quantized prediction error coefficients, and 

- determining a decoded piece of current video information using the piece of 
prediction video information, wherein the selected inverse motion coefficient 
quantizer is determined based on the prediction error quantizer. 

35 

19. A method for decoding encoded video information according to claim 15, 
wherein the encoded video information comprises information indicating the 
selected motion coefficient quantizer. 
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20. A method for decoding encoded video infonnation according to claim 15, 
further comprising a step of receiving signalling information indicating the selected 
motion coefficient quantizer. 

5 

21. An encoder for performing motion compensated encoding of video 
information, comprising: 

- means for receiving a piece of current video information, 

- memory means for storing a piece of reference video information, 

10 - motion field estimation means for estimating a motion field of picture elements in 
the piece of current video information using at least the piece of reference video 
information, 

- motion field coding means, which comprise 

- means for producing motion coefficients describing the estimated motion 
15 field, 

- first selection means for selecting a quantizer from a set of quantizers, said 
first selection means having an input to receive information indicating a 
selection criterion and an output to send information indicating the selecte4 
quantizer, and 

20 - quantization means for quantizing motion coefficients using the selected.;. ... 

quantizer, said quantization means having an input to receive information 
indicating the selected quantizer, a second input to receive the motion 
coefficients, and an output to send the quantized motion coefficients, and 

- motion compensated prediction means, which means comprise 

25 - second selection means for selecting an inverse quantizer from a set of 

inverse quantizers, said second selection means having an input to receive 
infonnation indicating a selection criterion and an output to send information 
indicating the selected inverse quantizer, 

- inverse quantization means for inversely quantizing the quantized motion 
30 coefficients using the selected inverse quantizer, said quantization means 

having an input to receive the quantized motion coefficients, a second input to 
receive information indicating the selected inverse quantizer and an output to 
send the inverse quantized motion coefficient, and r 

- means for determining a a piece of prediction video information using at least 
35 the piece of reference video information and the inverse quantized motion 

coefficients. 
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22. An encoder for performing motion compensated encoding of video 
information according to claim 21, wherein the information indicating the selection 
criterion indicates a value of a certain parameter used in the encoder. 

5 23. An encoder for performing motion compensated encoding of video 
information according to claim 22, comprising further: 

- prediction error means for determining a piece of prediction error information 
between the piece of current video information and the piece of prediction video 
information and 

10 - prediction error coding means for coding the piece of prediction error video 
information and representing the piece of prediction error video information using 
certain prediction error coefficients, which comprises second quantization means for 
quantizing the prediction error coefficients using a prediction error quantizer, 
wherein the second quantization means have an output to send information 

15 indicating the prediction error quantizer and the selection in the first selection 
means is based on the information indicating the prediction error quantizer. 

24. An encoder for performing motion compensated encoding of video 
information according to claim 21, wherein the information indicating the selection 

20 criterion indicates a certain image quality. 

25. An encoder for performing motion compensated encoding of video 
information according to claim 21, wherein the information indicating the selection 
criterion indicates a certain bit rate. • 

25 i 

26. A decoder for performing the decoding of encoded video information, 
comprising: 

- memory means for storing a piece of reference video information, 

- input means for receiving quantized motion coefficients, and 
30 - motion compensated prediction means, which comprise 

- selection means for selecting an inverse quantizer from a set of inverse 
quantizers, said second selection means having an input to receive information 
indicating a selection criterion and an output to send information indicating the 
selected inverse quantizer, 
35 - inverse quantization means for inversely quantizing the quantized motion 

coefficients using the selected inverse quantizer, said quantization means 
having an input to receive the quantized motion coefficients, a second input to 
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receive information indicating the selected inverse quantizer and an output to 
send the inverse quantized motion coefficient, and 

- prediction means for determining a piece of prediction video information 
using at least the piece of reference video information and the inverse 

5 quantized motion coefficients. 

27. A decoder for performing the decoding of encoded video information 
according to claim 26, wherein said encoded video information comprises 
information indicating said selection criterion. 

10 

28. A decoder for performing the decoding of encoded video information 
according to claim 26, further comprising means for receiving signalling 
information indicating said selection criterion. 

15 29. A decoder for performing the decoding of encoded video information 
according to claim 26, wherein said encoded video information comprises quantized 
prediction error coefficients, comprising 

- an input to receive the quantized prediction error coefficients, 

- means for detennining a prediction error quantizer used in the quantization of 
20 the prediction error coefficients and a corresponding inverse prediction error 

quantizer, and 

- second inverse quantization means for inversely quantizing the quantized 
prediction error coefficients using the inverse inverse prediction error 
quantizer, said second inverse quantization means having an output to send 

25 information indicating the prediction error quantizer, and 

the selection in the selection means is based on the information indicating the 
prediction error quantizer. 

30. Computer program element for performing motion compensated encoding of 
30 video information, comprising: 

- means for receiving a piece of current video information, 

- memory means for storing a piece of reference video information, 

- motion field estimation means for estimating a motion field of picture elements in 
the piece of current video information using at least the piece of reference video 

35 information, 

- motion field coding means, which comprise 

- means for producing motion coefficients describing the estimated motion 
field, 



WO 01/54417 



PCT/FI01/00044 



29 

- first selection means for selecting a quantizer from a set of quantizers, said 
first selection means having an input to receive information indicating a 
selection criterion and an output to send information indicating the selected 
quantizer, and 

5 - quantization means for quantizing motion coefficients using the selected 

quantizer, said quantization means having an input to receive information 
indicating the selected quantizer, a second input to receive the motioii 
coefficients, and an output to send the quantized motion coefficients, and 

- motion compensated prediction means, which means comprise 

10 - second selection means for selecting an inverse quantizer from a set of 

inverse quantizers, said second selection means having an input to receive 
information indicating a selection criterion and an output to send information 
indicating the selected inverse quantizer, 

- inverse quantization means for inversely quantizing the quantized motion 
15 coefficients using the selected inverse quantizer, said quantization means 

having an input to receive the quantized motion coefficients, a second input to 
receive information indicating the selected inverse quantizer and an output to 
send the inverse quantized motion coefficient, and 

- means for determining a piece of prediction video information using at least 
20 the piece of reference video information and the inverse quantized motion 

coefficients. J 

#■ 

31. Computer program element for performing the decoding of encoded video 
information, comprising: 
25 - memory means for storing a piece of reference video information, 

- input means for receiving quantized motion coefficients, and 

- motion compensated prediction means, which comprise 

- selection means for selecting an inverse quantizer from a set of inverse 
quantizers, said second selection means having an input to receive information 

30 indicating a selection criterion and an output to send information indicating the 

selected inverse quantizer, 

- inverse quantization means for inversely quantizing the quantized motion 
coefficients using the selected inverse quantizer, said quantization means 
having an input to receive the quantized motion coefficients, a second input to 

35 receive information indicating the selected inverse quantizer and an output to 

send the inverse quantized motion coefficient, and "■ 
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- prediction means for determining a piece of prediction video information 
using at least the piece of reference video information and the inverse 
quantized motion coefficients. 

5 32. Computer program element as specified in claim 30 or 31, embodied on a 
computer readable medium. 



WO 01/54417 



PCT/FI01/00044 



1/7 



10 

\ 



I n (x,y) E n (x,y) 

A 

14 



PREDICTION ERROR 
CODING 



13 



12 



11 



15 



PREDICTION ERROR 
DECODING 



MOTION 
COMPENSATED 
PREDICTION 



MOTION FIELD 
CODING 



MOTION FIELD 
ESTIMATION 



FRAME MEMORY 



'N | 

ATED U 
ION I 



R n (x,y) 



O 

z 

X 
I 

a. 
J— 



/M 

16 



FIG. 1 PRIOR ART 



WO 01/54417 



PCT/FI01/00044 



2/7 



20 

/ 



3 



CD 
2 

a. 



LU 
Q 



22 



/ 



PREDICTION ERROR I g " ( * ,:> »Q 



DECODING 



LA 



23 




24 



FRAME MEMORY 



P n (x,y) 



MOTION 
COMPENSATED 
PREDICTION 



FIG. 2 PRIOR ART 



< — | — x — I — x — I — x — J — x — I X — j — x — I — x — I X- 

n 1 1 1 1 1 1 1 r~ 

-4q 1 -3q t -2q, -q, 0 q, 2q^ 3q 1 4q^ 



\ X 1 X ( X 1 X ( X 1 X 

1 1 1 r— r 1 — ► 

-2q 2 -q 2 0 q 2 2q 2 3q 2 



-X 1 X J -x\ 1 X 1 X- 

_j 1 1 : 1 ; r 

-2q 3 -q 3 0 q 3 2q 3 



-3q, 



FIG. 3 



WO 01/54417 



PCT/FI01/00044 



3/7 



501 



Define a set of quantizers 



502 



Define quantizer selection criterion 



503 



<- 



Receive current frame IJ&y) 



504 



Estimate and model the motion of picture elements 
(Ax(x,y),Ay(x,yy) resulting in coefficients a v b. 



505 



Select quantizer 



506 



v Quantize motion coefficients a., b. resulting in a., fc) \ 



507 



Determine prediction frame 

P n (x>y) = RJix+lxCx^^+Ky^y)) using a., S t 



508 



Determine prediction error frame 
EJ&y) =I n (x,y>-P n Cx f y) 



509 
\i4 



Encode prediction error frame E n (x,y) 
and present it with coefficients c. 



510 



Quantize prediction error coefficients fa 
using second quantizer resulting in c. 



511 



Transmit coefficients a., c. 



512 



Transmit information about selected quantizer jj 




Target image 
quality 



Target bit rate 



Prediction error 
quantizer 



FIG. 5 



WO 01/54417 



PCT/FI01/00044 



4/7 



601 



Define a set of inverse quantizers 



602 



AAA 



Receive coefficients a v b v c. 



603 i 



Determine selected inverse quantizer 




604 

V 



^ Deter mine moti on coefficients a i , b t using g f , b t | 



605 



Determine basis functions /., g. 



606 



T 



Determine motion of picture elements 
(Ax(Xy),Zy(x,y)) using a t , b t 



607 



Determine prediction frame 

PJ&y) = R n (x+Ax(x,y)^y+Ay(x,y)) 



608 



Determine second coefficients c f using c. 



609 

v 



Determine decoded prediction error frame 
E a (x,y) using c t 



610 



Determine decoded current frame 
!jx,y) = P n (x,y)+E n (xj) 



FIG. 6 



WO 01/54417 



PCT/FI01/00044 



5/7 



13' 721' 



52 



MOTION 
COMPENSATED 
PREDICTION 



MOTION VECTOR 
FIELD DECODING 



Ax(x } y) = 'La i f i (x,y) 
£y(x,y) =2,b.g i (x,y) 



4 \1 



43 



\ INVERSE 
QUANTIZER 
SELECTION 



SET OF 
INVERSE 
QUANTIZERS 



40 



41 



|\ INVERSE 
QUANTIZATION 



a-.b. 



SELECTED 
INVERSE 
QUANTIZER 



12' 



MOTION FIELD CODING 



34 

N 



N QUANTIZER 
SELECTION 
QRITERION 



TARGET 
BIT RATE 



TARGET 
IMAGE 
QUALITY 



32 



[\ QUANTIZER 
SELECTION 



33 



SET OF 
QUANTIZERS 



30 



\ QUANTIZATION 



31 



A A 



SELECTED 
QUANTIZER 



51 



a t ,b t 



MOTION VECTOR 
FIELD CODING 



Ax(x,y) = 'La.f i (x,y) 
Ay(x,y) = S b.g. (x,y) 



(Ax(x,y),Ay(x,y)) 



FIG. 7 



WO 01/54417 



PCT/FI01/00044 



6/7 



14 



10' 



I n (x,y) E n (x,y) 



PREDICTION ERROR 
CODING 



current 
QP 



15 



17 



PREDICTION ERROR 
, DECODING 


: K 







V 



CD 

z 
—1 

Q. 



\ FRAME MEMORY 



13' 



MOTION 
k COMPENSATED 
42 PREDICTION 40 



select 



selected 

q- 



I 



I 



select selected 

q- I I q- 

32 MOTION FIELD 30 
CODING 



1^ 



MOTION FIELD 
ESTIMATION 



R n (x,y) 



/M 

16 



FIG. 8 



— I x — h 

n 



-X- 



-j X 1 X 1 Xr 



2q 



3q 



-3q 



.2q 



-q 



i 

o 



FIG. 4 



WO 01/54417 



PCT/FI01/00044 



3' 



CD 
Z 

B 

—i 
CL 

ID 



Ul 
Q 



7/7 



20' 



22 



PREDICTION ERROR | ^'^Q 



DECODING 



24 



FRAME MEMORY 



select 



42 



selected! 40 
q. 



MOTION 
COMPENSATED 
PREDICTION 



FIG. 9 



12* 



motion vector field 
of previous segments 



segment information 



(Ax(x,y), ±y(x,y)) 



61 



62 



1/ 



▼ ▼ ▼ ▼ ▼ 



Prediction Motion Field 



32 



QP in prediction 
error coding 



signalling 
information 
for pred. motion 
vector field 



{^x ptd (x,y),isy pid (x,y)) 



Motion 
analyser 



M 



3d 



Motion 
Coeff. 
Removal 



63 



MOTION FIELD CODING 



Select 
quantizer 



Quantization 



Current 



coeff. selection information 



FIG. 10 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/FI 01/00044 



A. CLASSIFICATION OF SUBJECr MATTER 
IPC7: H04N 7/36 

According to International Patent Classification (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 

IPC7: H04N 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 

SE,DK,FI,N0 classes as above 



Electronic data base consulted during the international search (name of data base and, where practicable, search terms used) 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



WO 9740628 Al (NOKIA MOBILE PHONES LTD.), 
30 October 1997 (30.10.97), page 8, 
line 1 - page 14, line 30; page 28, 
line 7 - line 21 



US 5396292 A (MURATA), 7 March 1995 (07.03.95), 
abstract 



WO 9717797 A2 (DAVID SARNOFF RESEARCH CENTER, 

INC.), 15 May 1997 (15.05.97), the whole document 



US 5778192 A (SCHUSTER ET AL.), 7 July 1998 
(07.07.98), the whole document 



1,4 



| x| Further documents are listed in the continuation of Box C. | y\ See patent family annex. 



* Special categories of cited document* 

* A* documail defining the general stale of the art which nol coasidcred 

to be of particular relevance 
*E* earlier application or patent but published on or after the international 
filing date 

*L' document which may throw doubts on priority claim(s) or which is 
cited to establish the publication date of another citation or other 
special reason (as specified) 

document referring to an oral disclosure, use, exhibition or other 



"O* 



document published prior to the international filing date but later than 
the priority date claimed 



*T* later document published after the international filing date or priority 
dale and nol in conflict with the application but cited to understand 
the principle or theory underlying the invention 

*X* document of particular relevance: the claimed invention cannot be 
considered novel or cannot be considered to involve an inventive 
step when the document is taken alone 

* Y* document of particular relevance: the claimed invention cannot be 
considered to Involve an in vena" vo step when the document is 
combined with one or more other such documents, such combination 
being obvious to a person skilled in the art 

*&* document member of the same patent family 



Dale of the actual completion of the international search 

14 Mav 2001 


Date of mailing of the international search report 

21 -05- 2001 

i 


Name and mailing address of the ISA/ 
Swedish Patent Office 
Box 5055, S-102 42 STOCKHOLM 
Facsimile No. + 46 8 6G6 02 86 


Authorized officer 

Erik Velllas /OGU 

Telephone No. + 46 8 782 25 00 



Form PCT/IS A/210 (second sheet) (July 1998) 



INTERNATIONAL SEARCH REPORT 



International application No. 

PCT/FI 01/00044 



C (Continuation). DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



WO 9638006 Al (MOTOROLA INC.), 28 November 1996 
(28.11.96), page 4, line 26 - line 35; page 5, 
line 25 - line 30 



US 5025482 A (MURAKAMI ET AL.), 18 June 1991 
(18.06.91), column 3, line 45 - line 55 



Form PCT/ISA/210 (continuation of second sheet) (July 1998) 



INTERNATIONAL SEARCH REPORT 

Information on patent family members 

02/04/01 


International application No. 

PCT/FI 01/00044 


Patent document 
cited in search report 


Publication 
date 


Patent family 
member(s) 


Publication 
dale 



wo 



9740628 Al 30/10/97 



AU 5501296 A 

EP 0894403 A 

JP 2000508127 T 



12/11/97 
03/02/99 
27/06/00 



US 5396292 A 07/03/95 CA 2090206 A 27/08/93 

JP 2894067 B 24/05/99 

JP 5244583 A 21/09/93 



WO 9717797 A2 15/05/97 JP 2000511366 T . 29/08/00 

US 6084908 A 04/07/00 



US 5778192 A 07/07/98 NONE 



W0 9638006 Al 28/11/96 AU 5372496 A 11/12/96 

US 5757668 A 26/05/98 



US 


5025482 A 


18/06/91 


AU 


621753 B 


19/03/92 








AU 


5575890 A 


29/11/90 








CA 


2017384 A,C 


24/11/90 








DE 


69015695 D,T 


11/05/95 








EP 


0399487 A,B 


28/11/90 








SE 


0399487 T3 










FI 


98681 B,C 


15/04/97 








FI 


902550 D 


00/00/00 








JP 


2308671 A 


21/12/90 








JP 


2520306 B 


31/07/96 








NO 


303479 B 


13/07/98 








NO 


902302 D 


00/00/00 



Form PCT/ISA/210 (patent family annex) (July 1998) 



